The present invention relates to spectral imaging in general and, more particularly, to spectral bio-imaging of the eye which can be used for non-invasive early detection and diagnosis of eye diseases and for detection of spatial organization, distribution and quantification of cellular and tissue natural constituents, structures and organelles, tissue vitality, tissue metabolism, tissue viability, etc., using light reflection, scattering and emission, with high spatial and spectral resolutions.
A spectrometer is an apparatus designed to accept light, to separate (disperse) it into its component wavelengths and measure a spectrum, that is the intensity of the light as a function of its wavelength. An imaging spectrometer (also referred to hereinbelow as a spectral imager) is one which collects incident light from a scene and measures the spectra (in part or in full) of each pixel or picture element thereof.
Spectroscopy is a well known analytical tool which has been used for decades in science and industry to characterize materials and processes based on the spectral signature of chemical constituents. The physical basis of spectroscopy is the interaction of light with matter. Traditionally, spectroscopy is the measurement of the light intensity emitted, transmitted, scattered or reflected from a sample, as a function of wavelength, at high spectral resolution, but without any spatial information.
Spectral imaging, on the other hand, is a combination of high resolution spectroscopy and high resolution imaging (i.e., spatial information). The closest work so far described with respect to the eye concerns either obtaining high spatial resolution information, yet providing only limited spectral information, for example, when high spatial resolution imaging is performed with one or several discrete band-pass filters [see, for example, Patrick J. Saine and Marshall E. Tyler, Ophthalmic Photography, A textbook of retinal photography, angiography, and electronic imaging, Butterworth-Heinemann, Copyright 1997, ISBN 0-7506-9793-8, p. 72], or alternatively, obtaining high spectral resolution (e.g., a full spectrum), yet limited in spatial resolution to a small number of points of the eye or averaged over the whole sample [See for example, Delori F. C., Pfilbsen K. P., Spectral reflectance of the human ocular fundus, Applied Optics Vol. 28, pp. 1061-1077, 1989].
Conceptually, a spectral imaging system consists of (i) a measurement system, and (ii) an analysis software. The measurement system includes all of the optics, electronics, illumination source, etc., as well as calibration means best suited for extracting the desired results from the measurement. The analysis software includes all of the software and mathematical algorithms necessary to analyze and display important results in a meaningful way.
Spectral imaging has been used for decades in the area of remote sensing to provide important insights in the study of Earth and other planets by identifying characteristic spectral absorption features. However, the high cost, size and configuration of remote sensing spectral imaging systems (e.g., Landsat, AVIRIS) has limited their use to air and satellite-born applications [See, Maymon and Neeck (1988) Proceedings of SPIE--Recent Advances in Sensors, Radiometry and Data Processing for Remote Sensing, 924, pp. 10-22; Dozier (1988) Proceedings of SPIE--Recent Advances in Sensors, Radiometry and Data Processing for Remote Sensing, 924, pp. 23-30]
There are three basic types of spectral dispersion methods that might be considered for a spectral bio-imaging system: (i) spectral grating and/or prism, (ii) spectral filters and (iii) interferometric spectroscopy.
In a grating/prism (i.e., monochromator) based systems, also known as slit-type imaging spectrometers, such as for example the DILOR system: [see, Valisa et al. (Sep. 1995) presentation at the SPIE Conference European Medical Optics Week, BiOS Europe '95, Barcelona, Spain], only one axis of a CCD (charge coupled device) array detector (the spatial axis) provides real imagery data, while a second (spectral) axis is used for sampling the intensity of the light which is dispersed by the grating as function of wavelength. The system also has a slit in a first focal plane, limiting the field of view at any given time to a line of pixels. Therefore, a full image can only be obtained after scanning the grating or the incoming beam in a direction parallel to the spectral axis of the CCD in a method known in the literature as line scanning. The inability to visualize the two-dimensional image before the whole measurement is completed makes it impossible to choose, prior to making a measurement, a desired region of interest from within the field of view and/or to optimize the system focus, exposure time, etc. Grating based spectral imagers are popular in use for remote sensing applications, because an airplane (or satellite) flying over the surface of the Earth provides the system with a natural line scanning mechanism.
It should be further noted that slit-type imaging spectrometers have a major disadvantage since most of the pixels of one frame are not measured at any given time, even though the fore- optics of the instrument actually collects incident light from all of them simultaneously. The result is that either a relatively large measurement time is required to obtain the necessary information with a given signal-to-noise ratio, or the signal-to-noise ratio (sensitivity) is substantially reduced for a given measurement time. Furthermore, slit-type spectral imagers require line scanning to collect the necessary information for the whole scene, which may introduce inaccuracies to the results thus obtained.
Filter based spectral dispersion methods can be further categorized into discrete filters and tunable filters. In these types of imaging spectrometers the spectral image is built by filtering the radiation for all the pixels of the scene simultaneously at a different wavelength at a time by inserting in succession narrow band filters in the optical path, or by electronically scanning the bands using AOTF or LCTF (see below).
Similarly to the slit type imaging spectrometers equipped with a grating, as described above, while using filter based spectral dispersion methods, most of the radiation is rejected at any given time. In fact, the measurement of the whole image at a specific wavelength is possible because all the photons outside the instantaneous wavelength measured are rejected and do not reach the CCD.
Tunable filters, such as acousto-optic tunable filters (AOTFs) and liquid-crystal tunable filter (LCTFs) have no moving parts and can be tuned to any particular wavelength in the spectral range of the device in which they are implemented. One advantage of using tunable filters as a dispersion method for spectral imaging is their random wavelength access; i.e., the ability to measure the intensity of an image at a number of wavelengths, in any desired sequence without the use of a mechanical filter wheel. However, AOTFs and LCTFs have the disadvantages of (i) limited spectral range (typically, .lambda..sub.max =2.lambda..sub.min) while all other radiation that falls outside of this spectral range must be blocked, (ii) temperature sensitivity, (iii) poor transmission, (iv) polarization sensitivity, and (v) in the case of AOTFs an effect of shifting the image during wavelength scanning.
All these types of filter and tunable filter based systems have not been used successfully and extensively over the years in spectral imaging for any application, because of their limitations in spectral resolution, low sensitivity, and lack of easy-to-use and sophisticated software algorithms for interpretation and display of the data.
No literature has been found by the inventors of the present invention describing high resolution spectroscopy combined with high resolution imaging applied to the eye.
A method and apparatus for spectral analysis of images which have advantages in the above respects were disclosed in U.S. Pat. Nos. 5,539,517 and 5,835,214 to Cabib et al., which are incorporated by reference as if fully set forth herein, with the objective to provide methods and apparatuses for spectral analysis of images which better utilizes all the information available from the collected incident light of the image to substantially decrease the required frame time and/or to substantially increase the signal-to-noise ratio, as compared to the conventional slit- or filter type imaging spectrometers and do not involve line scanning.
According to this invention, there is provided a method of analyzing an optical image of a scene to determine the spectral intensity of each pixel thereof by collecting incident light from the scene; passing the light through an interferometer which outputs modulated light corresponding to a predetermined set of linear combinations of the spectral intensity of the light emitted from each pixel; focusing the light outputted from the interferometer on a detector array, scanning the optical path difference (OPD) generated in the interferometer for all pixels independently and simultaneously and processing the outputs of the detector array (the interferograms of all pixels separately) to determine the spectral intensity of each pixel thereof.
This method may be practiced by utilizing various types of interferometers wherein the OPD is varied to build the interferograms by moving the entire interferometer, an element within the interferometer, or the angle of incidence of the incoming radiation. In all of these cases, when the scanner completes one scan of the interferometer, the interferograms for all pixels of the scene are completed.
Apparatuses in accordance with the above features differ from the conventional slit- and filter type imaging spectrometers by utilizing an interferometer as described above, therefore not limiting the collected energy with an aperture or slit or limiting the incoming wavelength with narrow band interference or tunable filters, thereby substantially increasing the total throughput of the system.
Thus, interferometer based apparatuses better utilize all the information available from the incident light of the scene to be analyzed, thereby substantially decreasing the measuring time and/or substantially increasing the signal-to-noise ratio (i.e., sensitivity).
Consider, for example, the "whisk broom" design described in John B. Wellman (1987) Imaging Spectrometers for Terrestrial and Planetary Remote Sensing, SPIE Proceedings, Vol. 750, p. 140. Let n be the number of detectors in the linear array, m.times.m the number of pixels in a frame and T the frame time. The total time spent on each pixel in one frame summed over all the detectors of the array is nT/m.sup.2. By using the same size array and the same frame rate in a method according to the invention described in U.S. Pat. No. 5,539,517, the total time spent summed over all the detectors on a particular pixel is the same, nT/m.sup.2.
However, whereas in the conventional grating method the energy seen by every detector at any given time is of the order of l/n of the total, because the wavelength resolution is l/n of the range, in a method according to the invention described in U.S. Pat. No. 5,539,517 the energy is of the order of unity because the modulating function is an oscillating function (e.g., sinusoidal (Michelson) or a similar periodic function, such as the low finesse Airy function with Fabry-Perot) whose average over a large OPD range is 50%. Based on the standard treatment of the Fellgett advantage (or multiplex advantage) described in interferometry textbooks [for example, see, Chamberlain (1979) The principles of interferometric spectroscopy, John Wiley and Sons, pp. 16-18 and p. 263], it is possible to show that devices according to this invention have measurement signal-to-noise ratios which are improved by a factor of n.sup.0.5 in the cases of noise limitations in which the noise level is independent of signal (system or background noise limited situations) and by the square root of the ratio of the signal at a particular wavelength to the average signal in the spectral range, at wavelengths of a narrow peak in the cases the limitation is due to signal photon noise.
Thus, according to the invention described in U.S. Pat. No. 5,539,517, all the required OPDs are scanned simultaneously for all the pixels of the scene in order to obtain all the information required to reconstruct the spectrum, so that the spectral information is collected simultaneously with the imaging information.
Spectral bio-imaging systems are potentially useful in all applications in which subtle spectral differences exist between chemical constituents whose spatial distribution and organization within an image are of interest. The measurement can be carried out using virtually any optical system attached to the system described in U.S. Pat. No. 5,539,517, for example, an upright or inverted microscope, a fluorescence microscope, a macro lens, an endoscope or a fundus camera. Furthermore, any standard experimental method can be used, including light transmission (bright field and dark field), autofluorescence and fluorescence of administered probes, light transmission, scattering and reflection.
Fluorescence measurements can be made with any standard filter cube (consisting of a barrier filter, excitation filter and a dichroic mirror), or any customized filter cube for special applications, provided that the emission spectra fall within the spectral range of the system sensitivity.
Spectral bio-imaging can also be used in conjunction with any standard spatial filtering method such as dark field and phase contrast, and even with polarized light microscopy. The effects on spectral information when using such methods must, of course, be understood to correctly interpret the measured spectral images.
Reflection of visible light from the ocular fundus has been used for many years for research and for routine eye inspection by ophthalmologists. It is also the basis for recording the eye status of a patient for disease and treatment follow up, both as pictures on a camera film and as digital images in the computer memory.
In contrast, the spectral dependence of the light reflection from different regions of the fundus has been relegated only to research work. The reasons for these facts are (i) images are a very direct means of presenting information to a human being, because they are easily interpreted, compared and remembered by the human brain; (ii) spectral data are much less direct, are not immediately understandable, and to be useful they must usually undergo several layers of mathematical processing before they are related to the bio-physiological properties of the tissue in question; and (iii) there has been so far no affordable instrumentation available to collect and analyze spectral data from the fundus, which is easy to use, fast, and reliable for a research or clinical setting.
As a result, at present, the use of spectral information in many fields, and in particular in ophthalmology, is lagging enormously behind the imaging techniques.
Recently, Applied Spectral Imaging Ltd. of Migdal Haemek, Israel, has developed the SPECTRACUBE technology. The SPECTRACUBE technology is based on an interferometer based spectral imager and as such it combines spectroscopy and imaging to use the advantages of both. It collects spectral data from all the pixels of an image simultaneously so that, after appropriate processing, the important chemical composition of the studied object (related to its bio-physiological properties) can be mapped and visualized.
The SPECTRACUBE technology was employed for spectral (color) karyotyping which simplifies and improves the detection capability of chromosomal aberrations using fluorescence emission [see, Multicolor spectral karyotyping of human chromosomes. E. Schroeck et al., Science, 273, 494-497, 1996; Multicolor spectral karyotyping of mouse chromosomes. Marek Liyanage et al. Nature Genetics p. 312-315, 1996; Spectral Karyotyping. Yuval Garini, et al. Bioimaging 4, p. 65-72, 1996; Hidden chromosome abnormalities in haemotological malignancies detected by multicolor spectral Karyotyping. Tim Veldman, Christine Vignon, Evelin Schrock, Janet D. Rowley & Thomas Ried. Nature Genetics, April, 1997: 406-410.; Spectral Karyotyping: Chromosomes in Color. Turid Knutsen, Tim Veldman, Hesed Padilla-Nash, Evelin Schrock, Morek Liyanage, Thomas Ried. Applied Cytogenetics, 23(2) 1997, pp. 26-32.; and Early Experiences with SKY: A Primer for the Practicing Cytogenetic Technologist. Michele Shuster, Ulrike Bockmuhl, Susanne M. Gollin. Applied Cytogenetics, 23(2) 1997, pp. 33-37].
Diabetic retinopathy is a potentially devastating condition of the human vision system, that, in most cases, can be controlled with timely laser treatment [Ferris (1993) (commentary) JAMA 269:1290-1291]. The American Academy of Ophthalmology has suggested screening schedules to detect when patients develop clinical conditions which should be treated [Diabetic Retinopathy: American Academy of Ophthalmology Preferred Practice Patterns. San Francisco, Ca.: American Academy of Ophthalmology Quality of Care Committee Retinal Pane, American Academy of Ophthalmology, 1989].
However the suggested screening schedule is expensive, and for some individuals even the current expensive screening is not sufficient because patients occasionally develop severe retinopathy between scheduled examinations. In spite of this, it has been shown that this screening is cost effective [Javitt et al. (1989) Ophthalmology 96:255-64]. This work shows that a large amount of money could be saved in health care follow up, if high and low risk patients could be more effectively identified. Therefore, any method that could increase the accuracy and reduce the cost of screening for diabetic retinopathy would be of high clinical value.
Currently, the recommended screening evaluation for diabetic retinopathy includes a detailed retinal evaluation and, in selected cases, color retinal photography [Diabetic Retinopathy: American Academy of Ophthalmology Preferred Practice Patterns. San Francisco, Ca.: American Academy of Ophthalmology Quality of Care Committee Retinal Pane, American Academy of Ophthalmology, 1989]. Fluorescein angiography of the retina is routinely performed today, but it is invasive, unpleasant, and causes occasional deaths. Furthermore, the additional information obtained by fluorescein angiography does not help categorize patients into those who may benefit from immediate laser treatment and those who will not [Ferris (1993) (commentary) JAMA 269:1290-1].
The oxygen supply of the retina is provided by both the choroidal and retinal circulation. The choroid serves as the oxygen source for the photoreceptors in the avascular outer retina, whereas the retinal circulation plays a crucial role in maintaining the oxygen supply to the neural elements and nerve fibers in the inner retina. Because of the high oxygen needs of the retina, any alteration in circulation such as seen in diabetic retinopathy, hypertension, sickle cell disease, and vascular occlusive diseases results in functional impairment and extensive retinal tissue.
Noninvasive measurements of the oxygen saturation of blood in retinal vessels was first proposed by Hickham et al. [Hickham et al. (1963) Circulation 27:375] using a two-wavelength photographic technique (560 and 640 nm) for retinal vessels crossing the optic disk (the region where the optic nerve connects to the retina). A more advanced approach based on the three wavelength method of Pittman and Duling is presented in Delori (1988) Applied Optics 27:1113-1125.
The present invention is the first step towards showing the usefulness of spectral imaging in general and the SPECTRACUBE technology in particular, as a new tool for the analysis of the physiological state of various structures of the human ocular fundus and enhance the accuracy of diagnosis and prognosis of certain diseases which affect the eye.
The ability to collect data of physiological importance in a spatially organized way, to store them for later retrieval and to display them in an enhanced image mode for easy interpretation provides a new horizon in ophthalmology.
In recent years, optical imaging methods and instrumentation for analyzing the ocular fundus have been making important progress. In particular, the integration of the optical techniques with sophisticated image processing by computer, are becoming more and more popular, because they contribute to the visualization of the pathology in vivo and to the quantification of the structure and functions or dysfunction involved therewith. In addition, it improves comparisons across time and can prevent loss of information (see Practical Atlas of Retinal Disease and Therapy, edited by William R. Freeman, Raven press, New York, 1993, p.19). Besides digital imaging by CCD cameras in conjunction with a fundus camera, the modem instrumentation includes Scanning Laser Ophthalmoscope (SLO), Laser Tomographic Scanner (LTS) (see Practical Atlas of Retinal Disease and Therapy, edited by William R. Freeman, Raven press, New York, 1993, p.19), Optic Nerve Analysis systems (Ophthalmic Photography, edited by Patrick J. Saine and Marshall E. Tyler, Butterworth-Heinemann, 1997, p. 269) and others, each of which employs a different and unique technology.
Different imaging modes are also used, such as "Confocal Imaging" and "Indirect Imaging" modes, to highlight and emphasize different features of the ocular fundus (see Practical Atlas of Retinal Disease and Therapy, edited by William R. Freeman, Raven press, New York, 1993, pp. 20 and 21).
It is also well known that imaging at specific wavelengths of visible light and the use of infrared light yields different types of information on the ocular fundus because different wavelengths are absorbed and scattered differently by the various fundus layers, anatomic structures (retina, choroid, vessels, pigment epithelium, sclera, etc.) and different depths (see Practical Atlas of Retinal Disease and Therapy, edited by William R. Freeman, Raven press, New York, 1993, p. 23, and Ophthalmic Photography, edited by Patrick J. Saine and Marshall E. Tyler, Butterworth-Heinemann, 1997, p. 71-73; see also U.S. Pat. application No. 08/942,122, which is incorporated by reference as if fully set forth herein).
Fluorescein and Indocyanine Green Angiography are standard techniques used by the ophthalmologists for the analysis of vessels, blood flow, and related pathologies, and provide information critical to the diagnosis and treatment of eye diseases (Ophthalmic Photography, edited by Patrick J. Saine and Marshall E. Tyler, Butterworth-Heinemann, 1997, p. 261-263, and pp. 273-279 and Practical Atlas of Retinal Disease and Therapy, edited by William R. Freeman, Raven press, N.Y., 1993, pp. 25-29). Both of these tests use the intravenous injection of fluorescent dyes, which circulate in the blood, and allow the documentation of ocular vasculature and blood flow. Fluorescein Angiography (FA) is used for retinal vessels, while Indocyanine Green Angiography (ICG) has advantages for imaging the choroidal vessels, for example choroidal neovascularization (Ophthalmic Photography, edited by Patrick J. Saine and Marshall E. Tyler, Butterworth-Heinemann, 1997, p. 264, FIG. 7-34). Disadvantages of FA and ICG are that they require injection of a dye, which sometimes is dangerous, for example there is a requirement for screening patients for iodine allergy, and since the effect of the dye is dynamic, several images must be recorded between 2 and 60 minutes after injection.
Blood is a worse absorbent of infrared light than of visible light, so it is also useful to image features which are in the posterior layers or behind vessels or thin hemorrhages (see Ophthalmic Photography, edited by Patrick J Saine and Marshall E Tyler, Butterworth-Heinemann, 1997, p. 263, and the study by Elsner A E et al., Infrared imaging of sub-retinal structures in the human ocular fundus, Vision Res. 1996 Jan; 36(1): 191-205.).
In spite of what is known today on the wavelength dependence of fundus imaging of reflected white light, there is no commercial instrumentation which is used for imaging of choroidal vasculature and other features at or near the choroidal depth, based on the reflection of white light.
There is thus a widely recognized need for, and it would be highly advantageous to have methods of spectral bio-imaging of the eye which can be used for non-invasive early detection and diagnosis of eye associated diseases.
In particular, there is a widely recognized need for, and it would be highly advantageous to have an instrumentation and method capable of high quality imaging of choroidal vessels and similar tissues at or near the depth of the choroid, using high throughput spectral imaging instrumentation of white light reflection, or reflection imaging of successive monochromatic light illumination. Among the results of high throughput are that the system can be built with a reasonably small size and can measure at reasonably high speed, so that it is more efficient and therefore suitable for sale in larger quantities and as a consequence, at lower cost. Another important advantage of the present invention over present methods of choroidal imaging is the avoidance of systemic dye injection and all related complications, for the clinic performing the test and for the patient receiving it.